1. Technical Field
The present invention relates to an apparatus for processing panoramic images or the like which have been photographed in all directions.
2. Background Art
It is strongly desired to observe surrounding environments at a certain view point as if the viewer was at that site. Specifically, if a viewing direction for an image of scenery photographed during a trip can be freely varied to display scenery extending in the viewing direction, the viewer can feel as if he or she was looking around though he or she is not actually at that site.
If the viewing direction can be freely varied for an image such as that obtained using a fisheye lens or an omnidirectional image obtained using a reflection mirror, which image has a field of view much larger than that obtained by a normal camera lens, the feeling of being at the site is further improved.
Japanese Patent Laid-Open No. 9-62861 discloses an apparatus for storing panoramic images photographed around a view point moving while traveling through space as well as a relative-moving position from a certain point.
In reproducing an image, an image extending in a specified viewing direction is synthesized from the panoramic image stored in the apparatus, and the synthesized image is displayed. When the user indicates a next view point or line of sight, a next image is correspondingly invoked, computed, and displayed; this operation is repeated.
The view point is a position where an image is photographed, and the line of sight is a direction in which a viewer looks at the view point position.
FIG. 21 shows a process loop of image reproduction according to this conventional technique.
The following description follows this process loop.
S201 is a loop in which an image is repeatedly synthesized and displayed.
At S202, a view point and a line-of-sight direction for a traveling path are initially computed. After the start loop, the next view point position and line-of-sight direction to be displayed are computed and determined from a relative movement computed at S206, relative to the current view point and line-of-sight direction.
At S203, a panoramic image corresponding to the view point position determined at S202 is loaded.
At S204, an image extending in the line-of-sight direction determined at S202 is synthesized from the panoramic image.
At S205, the image synthesized at S204 is displayed.
At S206, a user views the image displayed at S205 to indicate movement of the view point and the line-of-sight direction in which a visual angle is moved.
At S207, the process returns to S201 to repeat processing from S202 to S206.
At S206, the user can command move forward, move backward, stop, clockwise rotation, counterclockwise rotation, upward rotation, downward rotation, enlargement, contraction, and end.
Additionally, by retaining history of the view point position and line-of-sight direction, this conventional apparatus can predict the next possible view point position and line-of-sight direction to be indicated by the user in order to load a required image in advance, thereby achieving fast display.
In this manner, for a video photographed in all directions during movement, the conventional apparatus repeatedly invokes, for each reproduction, images photographed at a view point position and in a line-of-sight direction dependent on the user""s command and synthesizes and displays the images, to provide the user with images that provide him or her with the feeling of being at the site.
The above described conventional apparatus, however, still has the following problems:
 less than 1 greater than  An image photographed while traveling through space is retained, and the user checks the reproduced image, indicates relative viewpoint and line-of-sight direction for the image to obtain a result, and then synthesizes the next image to be displayed. Thus, after the information on the view point and the line-of-sight direction has been provided, a required image is invoked from all stored images and then displayed after the image synthesis process. Consequently, the reproduction is delayed.
In particular, when panoramic images and relative-moving positions are stored at another site on the Internet and a communication means that is relatively slow in communicating data is used for reproduction, the reproduction is more significantly delayed.
 less than 2 greater than  When the next possible view point position and line-of-sight direction to be indicated by the user are predicted by retaining the history of the view point position and line-of-sight direction for the displayed image, the reproduction speed increases to some degree. When, however, the viewpoint position is moved in an unpredictable direction, new invocation of an image and new computations therefor are required to thereby reduce the reproduction speed to prevent the smooth passage of time for the reproduced image, thereby degrading the feeling of being at the site.
 less than 3 greater than  A position to which the viewpoint is to be moved is specified based on a relative position along the traveling path, so that the image can only be repeatedly moved forward and backward along the sequence of images photographed while traveling through space. Accordingly, the display of an image on an unwanted path cannot be deleted.
 less than 4 greater than  The position to which the view point is to be moved is specified based on a relative position along the traveling path, so that user does not know which way to take next time in an image photographed at a site visited by the user for the first time. As a result, the user must reach the target by repeating moving forward and backward as required.
It is an object of the present invention to provide an omnidirectional video output method that enables fast reproduction and provides users with the feeling of being at the site.
It is another object of the present invention to provide an omnidirectional-video output method that allows a user to easily input desired traveling path and line of sight to a terminal device to receive display of only images on a required path.
An omnidirectional video output method according to the present invention comprises transmitting image pickup position data representing a service area having omnidirectional images, from a supply center to a terminal device, and transmitting at least view point information of desired view point information and line-of-sight information from the terminal device to the supply center. When only the view point information is transmitted from the terminal device to the supply center, only image information for a traveling path desired by the terminal device is transmitted from the supply center to the terminal device, which then processes and outputs the information depending on the desired view point. When the view point and line-of-sight information is transmitted from the terminal device to the supply center, only image information for a traveling path desired by the terminal device is processed depending on the desired line of sight based on the view point and line-of-sight information received from the terminal device and is then transmitted from the supply center to the terminal device, which then outputs the information.
According to the configuration of the present invention, the amount of data in information between the supply center and the terminal device is reduced to increase the reproduction speed, thereby providing users with the feeling of being at the site.
This configuration also improves the input method for the terminal device to facilitate the operation of inputting the desired traveling path and line of sight.
In addition, when the terminal device indicates the desired view point to the supply center, the supply center presents image pickup position data for omnidirectional images to the terminal device, which receives, based on the image pickup position data, image information for a desired traveling path determined depending on the desired view point. Accordingly, the user can easily input the desired traveling path and can continuously view only images on the desired traveling path in such a manner that the reproduction time passes smoothly.
Further, if not only the view point information but also the line-of-sight information is transmitted from the terminal device to the supply center, the supply center extracts omnidirectional images from an omnidirectional image position file based on the view point information, subsequently processes the images based on the line-of-sight information, and then transmits only the required image to the terminal device for display. Consequently, the user can enjoy an all-image display that provides users with the feeling of being at the site, regardless of an arithmetic-operation process speed of the terminal device.